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Executive summary 


4 The Engineering and Physical Sciences Research Council (the Council) are 
one of six Research Councils which provide grants to meet the cost of specific 
research projects proposed by universities and other organisations. These grants 
may include funding for equipment which is then purchased by universities. In 
1997-98 the Council made around 1,900 grants totalling some £260 million of 
which around £38 million (15 per cent) was for research equipment. 


Bi The Council give considerable freedom and flexibility to recipients of grants 
to manage the resources provided. By its nature scientific research is uncertain. 
The Council recognise that projects may change as the research progresses, either 
as a consequence of discoveries by the investigator or by others. The equipment 
needed may then be different from that applied for initially. 


3 | Our examination of research grants covered the last four years. During this 
period there have been considerable efforts by the higher education sector and the 
Research Councils to improve procurement arrangements generally. In 1996 the 
Joint Procurement Policy and Strategy Group, made up of representatives of the 
higher education sector, produced a procurement strategy and has been working 
with the sector to implement that strategy. In September 1998 the group provided 
new guidance on procurement of equipment and, in October 1998, guidance on 
whole life costing. The Cabinet Office has also issued guidance to departments on 
procurement issues including “Better Quality Services” and “Efficiency in Civil 
Government Procurement” (both July 1998), setting out the scope for efficiency 
gains in procurement. 


4 | The purpose of our examination was to establish whether there was any 
scope for more effective procurement of research equipment funded by the Council 
and if so what the Council could do to encourage the further implementation of the 
recent guidance on best practice. We examined 68 grants in order to identify the 
scope for further improvement. Our methodology is set out in Appendix 1. 
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Findings 


5 | Our study produced findings in five areas: 


M@ Scrutiny of grant applications; 


M@ Equipment purchased at keen prices; 


M Overestimation of costs in grant applications; 


M@ Equipment that could have been purchased more cheaply; and 


M Combining equipment requirements to increase purchasing power. 


Scrutiny of grant applications 


G When researchers apply for a grant they are required to identify the 
resources needed. Proposals are scrutinised through a process of peer review by 
academic experts in the relevant field of research. When equipment is required as 
part of the research project, a specific element in the total research grant awarded 
is identified by the Council as being available for this purpose. At the end of the 
grant period the achievements are assessed by the Council in terms of the research 
quality, output and cost effectiveness. Although the Council reconciles the funds 
against the expenditure, it does not review whether the equipment specifically set 
out in the application was actually purchased and whether it was purchased at the 
price specified in the grant proposal. 


Some equipment is purchased at keen prices 


Ed Against the background of the procurement initiatives launched in the 
higher education sector, many universities have taken steps to improve their 
procurement arrangements. In our study of procurement in the English higher 
education sector we estimated that the sector had achieved savings of around 
£38 million in 1994-95, £53 million in 1995-96 and a further £42 million in 
1996-97 from these initiatives. In 20 of the 68 cases we examined, universities had 
purchased equipment at keen prices. These cases were usually where suppliers 
had been willing to accept lower than normal prices in order to support, or 
associate themselves and their equipment with, a prestigious area of research. 
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Overestimating costs in grant applications 


8 | Our examination showed that, in our selection of cases, on average the 
equipment purchased by the universities cost around 16 per cent less than the 
amount provided by the Council. Generally the universities use the funds released 
to purchase additional equipment for the benefit of the research being undertaken. 


9 | Overestimating, where it occurs, is due to various factors. First, at the grant 
proposal stage, it is difficult for universities to obtain best prices from suppliers 
when they are not in a position to place a firm order. Second, prices can fall while 
the Council are considering the application. Third, some estimates are based on 
informally obtained prices rather than through formal quotations. Fourth, 
sometimes a margin is added to equipment estimates to insure against the Council 
not awarding a grant for the full value of the equipment proposed. Fifth, 
universities normally include provision for VAT in their applications, even if the 
equipment is subsequently zero-rated for VAT and the funds are not claimed from 
the Council. Finally equipment needs may change or alternative solutions to the 
equipment needs may be identified as research progresses. 


EX} We found that where the initial estimates were better evidenced, such as 
being based on competitive quotations, the estimates proved more accurate. Of the 
68 cases we examined nearly half had not been fully funded by the Council. In the 
year ending July 1998 the Council found that the situation had improved and 
grants were reduced by 6 per cent overall and by 15 per cent for the equipment 
element of grant. The Council would prefer to fund fewer higher quality grant 
applications, but in full. Better evidencing of research equipment costs would be 
one element of providing a better quality grant application and would help ensure 
applicants had a greater chance of success. 


Equipment that could have been purchased more cheaply 


Eke In the 68 cases (total value £8.7 million) we examined, savings of around 
£0.6 million (7 per cent) could have, in our view, been achieved by better 
purchasing. This was mainly because greater use could have been made of 
competitive purchasing, additional discounts could have been claimed in some 
cases and there had been scope for more effective negotiations with suppliers. 


EF] 3=We found a number of areas where other best practice was not being 
applied and the advice of the universities’ procurement experts was not sought. In 
no cases were alternatives to outright purchase, such as leasing, formally 
considered, or whole life costing adopted. A whole life costing approach to 
procurement takes account of the total cost of an item, from determination of need 
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Conclusions 


through acquisition, running costs and eventual disposal. The adoption of this 
approach enables purchasers to understand more fully the costs involved and to 
make more effective comparisons between different procurement options. We also 
found no examples of the universities benchmarking the prices they were 
obtaining against those obtained by other universities or departments for similar 
research equipment. 


Combine equipment requirements to increase purchasing 
power 


EF} The Council and other Research Councils have opportunities to co-ordinate 
major research equipment purchases. Combining together to increase purchasing 
power can produce additional savings because it strengthens the purchasers’ 
negotiating position. It improves their ability to gain enhancements to the original 
equipment specification and offers suppliers the opportunity to reduce 
manufacturing and marketing costs. Except for some computer hardware 
purchases, we found no evidence of the Council or the universities co-ordinating 
purchases of major research equipment despite the potential benefits for doing so. 
Depending on the opportunities arising throughout the year, we estimate that 
additional savings of at least two per cent overall could also be obtained by 
co-ordinating major equipment purchases. 


EZ} Gains can be secured by improving the accuracy of equipment cost 
estimates and the efficiency of equipment procurement. All the deficiencies in 
procurement we identified can be remedied if the universities follow the basic best 
practice set out in the “Equipment Procurement” guidelines for the higher 
education sector produced in September 1998 by the Joint Procurement Policy and 
Strategy Group referred to in paragraph 3. This guidance was issued in response 
to an earlier National Audit Office study in Scotland on equipment procurement by 
the Scottish higher education sector. It was issued after we had examined the case 
studies. It provides advice on competitive tendering, the use of preferred suppliers, 
negotiating effectively with suppliers, co-ordinating purchasing and involving 
procurement specialists. 


EF} 3= We consider that the findings from our selection are generally indicative of 
the scope for improving the estimating for, and procurement of, equipment funded 
through grants. The case studies were selected to illustrate the potential scope for 
improvement but not in a way to allow statistical inferences to be made. If our 
findings are indicative of the potential gains across the whole of the Council’s 
funding of research equipment, improved procurement would equate to 


Scope for improving 
estimating and 
procurement 


Recommendation 1 


Recommendation 2 


Recommendation 3 


Recommendation 4 


Recommendation 5 
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£2.7 million a year. Improved estimating could yield £6 million a year for the 
Council to fund on approved research which might not otherwise have been 
funded. However, it should be noted that this is a redirection, rather than an 
increase, in university research expenditure. 


Figure 1p rr 


This figure shows how improvements in estimating and procurement release funds for investment in 
other research projects. 


Scope for improvement (%) of total cost Annual value (£ million) 
Estimating equipment costs 16 6 
Improved procurement: 
Competition/negotiations 5 } 
Seeking discounts 2 eee 
Co-ordination of purchases 2 0.8, but dependent on 


opportunities arising 


We recommend that the Engineering and Physical Sciences 
Research Council should: 


Encourage universities to improve their estimating of equipment costs by 
requiring proposals for equipment funding to be fully evidenced, while 
recognising, however, the effort required in preparing such proposals. 


Make applicants aware that, for research proposals of equivalent quality, those 
that have well supported and realistic assessments of equipment costs are more 
likely to be fully funded. 


Encourage universities to follow best procurement practice, including that set out 
in the “Equipment Procurement” guidelines for the higher education sector. 


Identify opportunities for co-ordinated purchasing between universities. 


Work with the other Research Councils and the Higher Education Funding 
Councils to continue to improve procurement and seek assurance that best 
procurement practice is being used and measures agreed to monitor progress in 


equipment procurement performance. 


Ely = The Council accept our recommendations. In conjunction with the other 
Research Councils and the Office of Science and Technology, they have set up a 
Working Group to examine the report and to determine, with representatives of 
the higher education sector, how best to carry forward the report’s 


recommendations. 
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Part 1: Introduction 


EEW The Council are one of six Research Councils which together provide 
around £630 million to universities and other higher education institutions to fund 
research. About £115 million of this is for the purchase of research equipment. 
Generally they provide their funding as part ofa larger grant awarded for a specific 
research project, although grants purely for equipment are also made. 


EF} The Council are the largest provider of research grants. In 1997 they 
approved around 1,900 grants totalling some £260 million and, in 1997-98, 
incurred expenditure on research grants of £227 million of which £38 million 
(17 per cent) was for the procurement of research equipment. In June 1998 they 
were managing some 4,000 “live” grants (Appendix 1, Figure 14) valued at 
approximately £675 million with equipment of some £102 million. 


EF] University research equipment is funded from a variety of sources. The 
Research Councils between them provide 40 per cent of total funding. The other 
funding sources are the Higher Education Funding Councils who provide around 
25 per cent, charities who provide almost 15 per cent; private industry or business 
providing around 10 per cent; and the European Community, government 
departments and donations providing around 10 per cent. 


The Office of Science and Technology allocate funds between the 
Research Councils and agree the framework within which they 
operate 


ERY Research funds are provided to the Research Councils through the Office of 
Science and Technology which is part of the Department of Trade and Industry 
(Figure 2). The role of the Office of Science and Technology is to develop and 
co-ordinate Government policy on science and technology both nationally and 
internationally. Their overall objective is to ensure that science and technology 
expenditure is directed towards making the maximum contribution to national 
economic performance and quality of life. They are responsible for allocating the 
funds between the Research Councils and agreeing the framework within which 
they operate. In 1997-98 they made a grant to the Council of £386 million to cover 
their grant programmes and administrative costs. 
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Flow eSeercSameas [Figure 2 ua 
research funding 


Financial flows 


Office of Science and Technology 


Annual grant of £386m to 
Engineering and Physical 
Sciences Research Council 


Engineering and Physical 
Sciences Research Council 


£227m a year in research grants of 
which £38m is for equipment 


Higher Education Institutions 


Responsibilities 


Allocating funds between the Research 
Councils. 

Agreeing the framework within which the 
Research Councils operate (via the 
Council's Management Statement, 
Financial Memoranda). 


Promoting and supporting high quality 
basic, strategic and applied research in 
engineering and the physical sciences. 
Ensuring that efficient and effective use is 
made of their resources. 


Identifying their equipment needs for 


each research project. 
; @ Carrying out the research. 
Equipment £38m a year ™ = Procuring and maintaining the equipment 
funded through the grant. 


(mainly universities) 


Source: National Audit Office 


The Council appraise proposals for research grant, monitor 
progress and review outcomes 


ER As a Non-Departmental Public Body the Council have an obligation to 
ensure that the research programme resources they distribute, including grants 
for research equipment, are used economically, efficiently and effectively. The 
Council discharge their obligations by appraising, through peer review, proposals 
for grant, monitoring progress and reviewing outcomes. Reliance is placed on the 
universities’ procurement procedures and controls as a means of purchasing the 
research equipment well. Up to now the Council have expected universities to 
follow the advice on procurement best practice from the Funding Councils and 
have not separately reviewed or monitored the usage or adequacy of the 
procurement arrangements in place at universities. These arrangements are 


examined in more detail in Part 3. 


EE3 By its nature scientific research is uncertain. The Council recognise that 
initial intentions may change as the funded research progresses and that planned 
equipment purchases may need to be modified. The Council give the grant holders 
a high level of freedom and flexibility in the way they manage the resources 
provided so as not to stifle research ideas. 
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Grants are made to promote and support high quality research 


The Council’s Research 
Programme areas 


Source: The Engineering and 
Physical Sciences Research 
Council 


~< 


EEd Grants are made to further the Council’s research objectives which are to 
promote and support high quality research and related postgraduate training in 
engineering and physical sciences. The Council divide their support for research 
into eight scientific and engineering programmes (Figure 3). 


———————EeEeEE 


Engineering: 
General Engineering 
Engineering for Manufacturing 
Engineering for infrastructure, the environment and 
healthcare 
Science: 
Mathematics 
Physics 
Chemistry 
Technology: 
Materials 
Information Technology & Computer Science 


ER] The Council define equipment as items costing more than £1,000. Items 
costing less are classified as consumables. Examples of research projects and 
equipment funded by the Council are shown in Figure 4. 


EE} Grants provide the funds needed to meet the cost of specific research 
projects which may include equipment. Some research activity focuses on high 
quality blue skies research to further scientific understanding and support for the 
British science base. In the more applied research areas, such as engineering, the 
research may be tied more to the potential relevance of the results of the research 
to industry and commerce. 


ER As with other Research Councils, the Council have a much greater demand 
for research funding than can be met within the resources available. Less than 
40 per cent of the applications they receive are successful and receive funding. 


Research projects funded 
by the Council 


Source: National Audit Office 
database of live research grants 
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|Figute 4 


Research proposal 


Research into the electromagnetic behaviour of 
future lightweight vehicles which use composite 
body panels instead of steel. 


Research into structures of single crystals using 
x-ray crystallography for the purpose of 
developing new materials. 


Research to promote the knowledge of the 
properties of solid surfaces and the development 
of the processes involved. 


Research into the unstable vibrations in vehicle 
brakes. 


Development of a facility for routine chemical 
analysis to enhance the throughput of routine 
samples. 


Research into practical uses of lasers for surface 
processing, including the development of a laser 
percussion hole drilling technique. 


Equipment required 


High quality graphic computers. Used to model 
the effects on vehicle electrical system of using 
composites. 


X-ray diffractometer. The diffraction of x-rays by 
the atoms or ions of a crystal enables information 
to be obtained on the structure of crystals. 
Research will aid the development of new 
materials. 


Scanning Transmission Microscope. An 
instrument to analyse the physical structure of 
surfaces It will provide a modern surface analysis 
Capability. 


Components for a test rig used to test the 
vibrations in vehicle brakes. 


Automated Mass Spectrometer. An apparatus 
used for separating isotopes, molecules, and 
molecular fragments by passing them through 
electric and magnetic fields. Used for analysing 
chemical samples. 


Laser. A device that amplifies light by stimulating 
emission of radiation. The laser’s beam of 
intensely pure light can cut through metal. 


A peer review process is used to appraise grants 


EREs The award process is described in Part 3 but Figure 5 outlines how 


applications are handled. 


Procurement of Equipment from Research Grants 


How research grant 
proposals are handled 


Referees 


Jal mice)nae elce) elessies 
A KWemicelaamexe) |(=te[=) 


Financial 


Rank order 
allocations 


Source: The Engineering and 
Physical Sciences Research 
Council 


Fifteen universities receive over half of the Council’s funding 


ERP] The Council provide grants for research by investigators in universities, 
and other similar institutions of higher education. Some 120 organisations are 
currently in receipt of research funding from the Council, although 15 universities 
receive over half of the funds for equipment (Figure 6). 


Our objective was to establish whether there was scope for more 
effective procurement of funded equipment 


ERFI We conducted this examination for a number of reasons, but mainly 
because: 


HM the equipment element of grants is a substantial (17 per cent) part of the 
Council’s expenditure; 


M the area offered potential for improvements and efficiency savings. The 
Council themselves were keen to make any cost effective improvements 
that would enhance value for money; and 
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Melnecinlentatet | Figure 6 
Council funding for 
equipment These 15 Higher Education Institutions receive around £58m of the £102m currently 


committed to equipment. 


Cambridge” 
Oxford 


Southampton 


Imperial College” 


Surrey* 


Liverpool* 


Sheffield* 


Warwick” 
Nottingham 


Leeds 


UMIST 


Manchester 


Birmingham 


Royal Institution 


Edinburgh 


Emillion 


Source: The National Audit Office 
database of live grants as at 
March 1998  Note:* The NAO visited these universities and the University of Glasgow. 


HM other studies by the National Audit Office in 1993, 1996 and currently on 
purchasing in the higher education sector have made recommendations 
on ways to improve procurement. The 1996 study focused on research 
equipment in Scotland. 


EE?y Our examination of research grants covered awards made by the Council 
over the last four years. During this period, the Higher Education Funding Councils 
in England, Scotland and Wales, the Department for Education (Northern Ireland) 
and the universities themselves have progressively introduced initiatives to 
improve procurement in universities. In 1993 the National Audit Office reported 
on University Purchasing in England and in 1994 the Committee of Public 
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Accounts subsequently reported on the progress that had been made. In 1996 
representative bodies of the higher education sector established a Joint 
Procurement Policy and Strategy Group to co-ordinate and direct all strategic 
activities aimed at promoting good procurement practice in the sector. In 1996 the 
group produced its procurement strategy for Higher Education and has been 
working with the sector to implement the strategy. In February 1999 the group set 
out its revised procurement strategy as a consultation document. It gives increased 
emphasis on providing support for implementing its guidance and is supported by 
draft guidelines on “Effective Procurement in Higher Education” and a model 
procurement strategy. The Cabinet Office has also issued guidance on 
procurement issues including “Better Quality Services” and “Efficiency in Civil 
Government Procurement” (both July 1998), setting out the scope for efficiency 
gains in procurement. 


EEE} The purpose of our examination was to establish whether there was any 
scope for more effective procurement of research equipment funded by the Council 
and, if so, to establish what the Council could do to promote the further 
implementation of the recent guidance on best practice. The two main issues 
examined were: 


M™ whether research equipment funded by the Council. could be purchased 
more cheaply; and 


M@ how the Council ensure that proposed equipment resources are justified, 
well procured and used for the purposes intended. 


ER Our methodology is set out in Appendix 1. In brief the National Audit Office 
selected six universities to visit from the 15 institutions receiving the highest 
proportion of their funds, plus one other university in Scotland. We selected and 
examined 68 grants to universities for case study, from which we selected, where 
possible, individual equipment purchases for price checking by our consultants. 
The grants were selected to cover all types of grant, different levels of equipment 
funding — from around £25,000 to over £1million - and a range of research 
equipment. The grants selected for study totalled £60 million of which £14 million 
was for equipment. The universities to whom the grants relate had been awarded 
grants from the Council totalling some £300 million over the last five years 
(Figure 7). 
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Value of grants awarded to universities visited by the National Audit Office 


Over the last five years the Council awarded to the seven universities visited a total of £300 million in grants. 


25.0 


BB cambridge [imperial College [BM Glasgow MM shettieia | 


al Surrey C Liverpool 2 Warwick 


SDS RE aN ee SR Na MeO ee Te 


20.0 


15.0 
£m 
10.0 
5.0 
1994* 1995 1996 1997 1998 
Year 


Note:* From 1st April 1994. 


Source: The Engineering and Physical Sciences Research Council's research grant statistics 


Another National Audit Office examination looked at the higher education 
sector’s progress in developing and implementing improvement in procurement 
since the 1993 National Audit Office report on University Purchasing in England 
(HC 635 of 1992-93). The results of the examination are set out in Procurement in 
the English higher education sector, (HC 437 of 1998-99). 


13 


Procurement of Equipment from Research Grants 


Part 2: Procurement of equipment funded 


by grant 


E30 This part of the report shows, on the basis of our examination, that 


although some equipment is purchased well, procurement performance overall 


could be better. In particular we highlight the low level of competitive 


procurement, the failure to take advantage of all available discounts and the 


benefits of better negotiation with suppliers. We also identify the possibility of 


taking greater advantage of opportunities to co-ordinate purchases of research 


equipment. The universities are not fully utilising the skills of their procurement 


advisors or identifying opportunities for better planning and co-ordination of their 


research equipment procurement activity. 


E2] This part sets out: 


how universities procure equipment; 


the results of our examination; 


procurement through competition; 


obtaining discounts; 


negotiating with suppliers; 


the use of co-ordinated procurement; and 


compliance with other aspects of best practice. 


Academic researchers are mainly responsible for procuring 


research equipment 


EX} The procurement arrangements in operation at universities vary but 


broadly follow the pattern set out in Figure 8. The key procurement decisions are 


taken by academic researchers who normally: 


write the research proposal; 


submit the application for grant to the Council (through the university); 


Procurement 
arrangements at 
universities 


Source: National Audit Office 
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M decide what equipment to purchase and how to buy it; and 


M carry out any competitive procurement and/or any negotiations with 
suppliers. 


Such decisions should be taken within the framework of the conditions the Council 
attach to the grant and the university's own financial and procurement 
regulations. All the universities we visited had a purchasing and supplies 
department with experienced procurement staff holding recognised procurement 
qualifications, a research grants section responsible for checking and monitoring 
grant applications, and a finance department who allocate the research budgets 
and account for the subsequent receipt of funds and charging of expenditure. 


Research proposal 


Research unit 


vetting 


Application to Council 
Equipment identified 


Proposal drafted Research funders 


Ronne anton 

to research 

department —|f approved, budget 
allocated 


University 
Department 


May get involved 


Single supplier 
Competitive tender Accounts Dept 


Dulfeee ea ae Preferred supplier 
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EZ] Within the higher education sector there are a number of inter-related 
organisations involved in procurement matters. They include: 


M the Funding Councils for England, Wales and Scotland and the 
universities’ Committee of Vice Chancellors and Principals, the Standing 
Conference of Principals and Committee of Scottish Higher Education 
Principals; . 


HM a Joint Procurement Policy and Strategy Group, established by the above 
bodies to co-ordinate and direct all strategic activities directed towards 
promoting good procurement practice in higher education institutions, 
including the development of guidelines for the procurement of 


equipment; 


M@ regional purchasing consortia and national and inter-regional working 
groups; and 


M professional procurement groups, such as the Association of University 
Purchasing Officers. 


EX] The universities have arrangements for purchasing many goods and 
services through purchasing consortia or regional purchasing organisations. But 
little use is made of them to purchase research equipment. This is because 
research equipment is regarded as specialised and the scope for obtaining savings 
via organisations which aggregate the purchasing power of many universities has 
been seen as limited. Priority has been given to lower cost higher volume items. 


Some equipment was purchased at keen prices, some equipment 
could have been purchased more cheaply 


EX3 For our selection of 68 grants we reviewed with the researchers 
responsible at the universities why and how they purchased the items funded by 
the grant. Where possible (62 cases) we commissioned an independent consultant 
to check whether the prices paid for the equipment were in line with market levels. 
It was not possible to price check six cases mainly because the selected items were 
being built from a number of smaller components. In seven further cases, our 
consultants encountered a lack of co-operation from research equipment 


manufacturers. 
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We found that in 20 cases the universities had purchased equipment at 
keen prices. These were usually where manufacturers/suppliers were willing to 
accept lower prices in order to support, or associate themselves and their 
equipment with, a prestigious area of research. This applied particularly where 
the work involved development of equipment which could have future commercial 
benefit to the manufacturer. An example of effective procurement is shown in 


case 1. 


Illustrative case 1 
A keen price 


Purchase of a scanning ion microscope (an 
electron microscope capable of forming a 
three-dimensional image on screen by means 
of a focused beam of electrons that is scanned 
across the object being viewed). It will be used 
to develop analysis, fabrication and 
modification of structures such as microchips. 
The equipment estimated to cost £385,000 was 


Illustrative case 2 


Selecting best prices 


Preliminary estimates were obtained, at 
proposal stage, for the supply of computer 
equipment. Further quotes were obtained 
before the start date of the grant and a third and 
fourth set obtained just before purchase. The 
purchaser eventually saved 23 per cent by 
selecting best prices for individual items from all 
the quotes. 


acquired for £245,000 with the aid of a 
substantial contribution from the manufacturer 
as programme collaborator. 


EX} 9 We and our consultants consider however that, in our selection of 68 cases 
(total value £8.7 million), on average, some seven per cent savings (£0.6 million) 
could be achieved by better purchasing (Figure 9). This was mainly because 
greater use could have been made of competitive purchasing, in some cases 
additional discounts could have been claimed and there had been scope for more 
effective negotiations with suppliers. Seven per cent equates to some £2.7 million 
for the Council’s equipment funding. Depending on the number of opportunities 
arising throughout the year, we estimate that additional savings of at least 
two per cent overall might also be obtained by co-ordinating major equipment 
purchases. The extent to which savings would be applicable to the other Research 
Councils depends on the thresholds they operate for requiring supporting invoices 
and on whether virement is permitted. 
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Analysis of results by 
categories of potential 
savings 


| Figuic 9 


< 


This analyses the results of the National Audit Office examination and their consultants price check, 
and shows three categories where scope for savings were identified. 


Category of potential savings Number' Scopefor Average % for items 
savings %of examined’ 
identified price 
(£) paid 
Items purchased at a keen price 20 - - - 
Competition not sought/negotiation with suppliers 48 248,400 5 } 6.9 
Discounts not sought 35 344,700 6.8 } 
Co-ordination of purchases not considered 2 180,0007 10 ze 
Total 773,100 8.9 
Notes: 1. Potential savings for one grant can occur in more than one category. 
2 Relates to two specific items within the National Audit Office selection. 
3 The total cost of the items examined was £8.7 million. 


There is scope for more competitive tendering 
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PX} The Joint Procurement Policy and Strategy Group (paragraph 2.4) consider 
that the primary basis for providing evidence to the taxpayer that value for money 
is achieved is by procurement through competition. They believe that best value 
for money means accepting the best combination of cost and quality while also 
taking account of important criteria, such as, delivery, equipment compatibility, 
warranty and maintenance. This can best be achieved by competition. This view is 
confirmed in guidance issued by the Committee of University Chairmen, which 
states that, save in exceptional cases, all procurements should be undertaken on 
the basis of full and open competition. The Council state in their grant guidelines 
that, although contract procedures are a matter for the organisations to whom 
they award grants, they expect that contracts will be let as a result of competitive 
tendering. 


EST) The main argument for not adopting competitive tendering was that there 
was only one supplier. We and our consultants, believe, however, that this is very 
rarely the case. Competition appeared to be feasible in over ninety per cent of the 
68 cases we examined, but we found that it had been applied in only 
14 (20 per cent) of cases, representing 34 per cent by value. In only three cases, 
where competitive tendering had not been undertaken, were we able to accept and 
confirm that only one supplier existed. The benefit of competition is demonstrated 
by case 3 while cases 4 to 6 show that there is scope for savings through greater use 
of more effective competition. 
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Illustrative case 3 
Good competition 


Mass spectrometer (an apparatus used for 
separating isotopes, molecules, and molecular 
fragments by passing it through electric and 
magnetic fields) which will be used for 
developing the study of molecular structures of 
synthetic materials. Estimated to cost 
£254,000. Obtained for £198,000 after 
competition, involving five potential suppliers. 


Illustrative case 4 
No competition 


Two central processing units were purchased 
without going to competition. According to our 
consultants, savings of at least 30 per cent 
could have been achieved by competitive 
tendering. 


PRET Of the 55 items price checked, our consultants concluded that, in 35 cases 
(64 per cent), the prices paid did not reflect normal market levels, and that 
additional savings averaging seven per cent could have been obtained. We and our 


consultants found several examples which demonstrated the benefits of effective 


competition and which addressed all aspects of value for money. 


Illustrative case 5 
Limited competition 


Quotes were obtained from two suppliers for a 
high performance mass spectrometer (an 
apparatus used for separating isotopes, 
molecules, and molecular fragments) costing 
around £85,000. It will be used primarily to 
study biological micro-structures of things 
such as proteins. Quotes received were 
similar but one of the suppliers was rejected 
without being given the opportunity to 
improve their offer as they had asked to do in 
their original response. 


Illustrative case 6 
Separating purchases for tendering 


Ordering complete systems of research 
equipment from the supplier of the main 
equipment does not necessarily guarantee 
good value for money because keen prices 
were often not given on the ancillary equipment. 
Our consultants advised us that savings of 
25-30 per cent can be achieved by placing 
separate orders for the peripheral equipment. 


PFI In deciding on whether to proceed with competitive tendering it is of course 
important to ensure that a balance between the costs of organising competition 
and the benefits likely to be gained is taken into account. There may also be 
advantage, once it is established that a supplier is offering good quality products at 
competitive prices, in entering into longer term or preferred supplier 


arrangements in order to minimise the costs of competition. 
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Not all available discounts are obtained 


PSF} When open competition is not possible, purchasers should take standard 
discounts on offer, or obtain special discounts, from suppliers. In 20 cases 
universities negotiated prices lower than were generally available because the 
equipment supplier wished to support the research or have their equipment 
associated with it. But in 35 cases discounts that were obtainable were not 
negotiated including available “university” discounts and discounts for prompt 
payment. Case 7 identifies a failure to achieve satisfactory discounts. But it is 
important to ensure that, as far as possible, the basic price, from which discounts 
are deducted, is real and verifiable, for example from published price lists. 


Illustrative case 7 
Prompt payment discount not secured 
A fibre optic camera part of an instrument used for measuring diffraction, which by applying a beam of 


electrons can determine the structure of crystals, costing £54,000. Our consultants advise that an 
additional 2 per cent discount for prompt payment, worth £1,150, was available but not taken. 


Negotiation with suppliers can achieve savings 
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PRP] There were examples where post tender or other negotiations with 
suppliers had achieved considerable savings or other benefits in kind from 
suppliers. None of the staff undertaking the negotiations were, however, trained in 
purchasing or contract negotiations. The staff undertaking the negotiations were 
the academic staff and there was no input from the universities’ professional 
procurement staff. As a result, in the examples where a supplier was selected 
non-competitively, we saw no evidence of them being asked to supply breakdowns 
of their prices to assist in subsequent price negotiations. Case 8 identifies the 
additional benefits that were gained from such negotiations. Case 9 identifies a 
failure to negotiate effectively over price or seek a breakdown of costs. 
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Illustrative case 8 Illustrative case 9 

Good purchasing negotiations Scope for negotiation 

Purchase of a laser marker (a device that Purchase of a prototype detector costing 
generates an intense light beam). A £158,000. The first of 16 detectors needed to build 
commercial laser system was needed to a new gamma-ray spectrometer for radioactive 
develop a new method for the production of beam spectroscopy necessary for the study of the 
rapid prototyping/tooling processes. The properties of nuclei. The university did not seek to 
department negotiated with the supplier after negotiate the price down, despite being in a good 
receiving four quotes. Following negotiations negotiating position. The prototype may lead to 
with the potential supplier a further reduction subsequent significant follow-on orders, possibly 
of 20 per cent was achieved realising in excess of £2 million, although the specification 
additional savings of £10,000. and funding for these detectors has yet to be 


finalised. A breakdown of the cost of building the 
prototype was not sought either. 


Combining equipment requirements to increase purchasing power 


PRE} Institutions have the opportunity to aggregate their equipment 
requirements to yield substantial savings on the purchase cost of equipment, and 
associated costs such as maintenance, delivery and procurement costs. Similarly, 
the universities, the Council and other Research Councils have the opportunity to 
co-ordinate major research equipment purchases. Improved planning by the 
universities can help to ensure that such opportunities are identified and joint 
procurement approaches initiated. In addition to possible financial savings, 
co-ordinated purchases increase the procurement power of the institutions 
involved and their ability to gain enhancements to the original equipment 
specification and other commercial advantages. The Council is in a position to 
implement the procurement practice of pursuing joint procurement strategies as 
laid down by the Government in its Comprehensive Spending Review report 
(CM 4011). 


EST Except for some computer equipment (Case 10), we found no evidence of 
universities co-ordinating the purchase of the same equipment. In one case (Case 
11) the Council approved funding for the purchase of similar specification Electron 
Microscopes by three universities in a three month period. The Higher Education 
Funding Council for England funded another through the Joint Research 
Equipment Initiative. No attempt was made either by the Council or the 
universities involved to co-ordinate these purchases to secure best value for 
money. Our advisers, the Société Genéralé de Surveillance, estimate that 
co-ordinating the purchase of the three machines may have saved an additional 
£242,000 (ten per cent). They informed us that, in their experience, savings of at 
least this level have been achieved in other major co-ordinated purchases. The 
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Higher Education Funding Council for England told us that they had secured 
substantial savings of around £3 million over two years by combining other orders 
under their Joint Research Equipment Initiative. 


Illustrative case 10 Illustrative case 11 

Co-ordinated purchase Potential co-ordinated purchase 

The computer requirements of two research The Council separately funded three Electron 
projects were combined to increase the Microscopes (instruments with high magnification 
university's purchasing power. Subsequent and resolution employing electron beams in place 
negotiations with the supplier obtained a of light, using electron lenses) costing between 
package representing a57 per cent reduction £728,000 and £1.1 million and the -Higher 
from the list price. Our price check confirms Education Funding Council for England funded 
that the university received a good deal from another. Each purchase was undertaken 
the supplier. separately, by four different universities, involving 


detailed assessment and testing of the 
specifications of the available microscopes. 


The Councils are in a position of knowing when grants for similar 
equipment are made to different universities and where there may be scope for 
these purchases to be co-ordinated. The Council do not maintain central records of 
the major research equipment items they fund. Such information would be useful 
to them and university procurement officers for identifying scope for co-ordinating 
purchases. To help universities exploit the opportunities to co-ordinate research 
equipment purchases, the Council need to consider how they could provide them 
with early notification of impending major purchases. The Council is considering 
disseminating such information via its website. 


PRT) In view of the high value and volume of research equipment being 
purchased by universities with Research Councils’ funds, further benefits could be 
gained from benchmarking prices and procurement processes to test whether they 
are meeting best practice. This does not have to be a complicated process. 
Universities could contact others who have bought similar equipment to ask them 
what price they had paid. We found however no evidence of procurement 
benchmarking being undertaken by the universities for major research 
equipment. During our discussions with researchers we identified a general 
reluctance to formally share information on prices achieved for purchasing 
research equipment although they claimed they were in touch with each other and 
with the market. Information on prices paid for similar items of research 
equipment was generally indicative rather than specific. 
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Other aspects of best practice 


Guidelines on equipment procurement have been 
published recently 


ERPI Despite the significant level of funds provided to universities by the Council 
for the purchase of research equipment they do not offer them comprehensive 
guidance on best practice and how to purchase equipment more expertly. Some of 
the universities we visited published their own guidance on purchasing though it 
varied in quality. 


Pet] In their 1993 Report on University Purchasing, the National Audit Office 
reported on the lack of procurement guidance in universities. In another Report in 
1996 the National Audit Office examined the Management of Teaching and 
Research Equipment in Scottish Higher Education Institutions. This was done 
partly in response to concerns expressed by the Scottish Higher Education 
Funding Council about the absence of guidance on equipment management. It 
concluded that institutions could do more to manage equipment in a structured 
way and to improve their purchasing practice. 


Px The representative bodies for the higher education sector took steps to 
remedy such deficiencies when, in 1996, the Joint Procurement Policy and 
Strategy Group developed and published a procurement strategy for the sector 
(paragraph 1.14). Since then central guidance to the higher education sector has 
been issued - including, for example, guidance on benchmarking, whole life 
costing and building and engineering projects. More relevant to research 
equipment, the Joint Procurement Policy and Strategy Group published 
“Equipment Procurement" guidelines (in September 1998) and “Whole Life 
Costing” guidance (October 1998) for the higher education sector. This was after 
we had completed our fieldwork for this study. This guidance is aimed particularly 
at academic staff wishing to purchase specialised equipment for teaching and 
research and provides comprehensive and practical advice on how to achieve 
successful institutional equipment procurement and represents accepted best 
practice. The guidance sets out best practice on all the procurement issues 
identified in this report. We believe that, if the universities followed the basic best 
practices set out in Figure 10, it would help them ensure they obtain better value 
for money in their procurement of equipment. 
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[Figure 10 | Procurement of equipment best practice 


These best practices represent the key issues that should be considered in purchasing equipment. 


Procurement process 
Need 


Specification 


Value analysis 


Co-ordinated purchases 


Equipment sharing 


Funding 


Supplier appraisal 


Competitive tendering 


Negotiations 


Selecting best value 


Equipment management 


Best practice 
The users decide what is needed and when. They should plan ahead for replacement of new 


equipment and endeavour to take account of the procurement implications of research needed to be 
funded by the Council in their overall strategy. 


Avoid insisting on compatibility of equipment at any price, requiring a higher specification than really 
needed, or specifying so narrowly that true competition is inhibited. 


Every feasible alternative should be considered. This can include upgrading existing equipment or 
purchasing a second hand or ex-demonstration model. 


Aggregation of similar equipment needs can bring substantial savings in the item cost, maintenance 
charges, delivery and in ordering and processing costs. 


The possibility that suitable equipment already exists in another department should be considered. 
Where equipment is purchased, consideration should be given to making it available for use by other 
departments. 


Reliable estimates of the cost of the equipment should be sought. Alternative forms of funding should 
also be considered, such as a leasing agreement or a private purchase initiative. 


The potential suppliers should be researched and assessed and competitive tendering used. 


The introduction of competitive tendering significantly increases an institution's purchasing power and 
negotiating position. Cases where there is a sole source of supply are very rare. The procurement 
office should be involved in the less technical aspect of the tender, including the requirements of the 
European Community procurement directives. 


Post tender negotiation can take place to obtain an improvement in the content of the tender. 
Procurement officers, with appropriate purchasing negotiating skills, should be involved. 


The selection of the best value tender should be evidenced and records kept on how the decision 
was reached and justified. Costs must take full account of all whole life costing considerations. 


The effective management of equipment involves formally monitoring usage, seeking to minimise 
running and maintenance costs and negotiating maintenance agreements at keen prices. 


Source: National Audit Office summary of the higher education guidelines for equipment procurement 
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The specialist skills of purchasing co-ordinators are not 
always used 


P22] Most research equipment is purchased by university departments who 
manage budgets delegated to them. They are best placed to draw up the often highly 
complex user specification of the equipment they want. But little recourse is made to 
the specialist skills which the universities' purchasing co-ordinators can bring to the 
procurement of the equipment. Best practice guidance recommends that appropriate 
use should be made of available specialist purchasing skills in all aspects of the 
procurement process particularly where high value capital equipment is required. 
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E51 Most of the universities we visited employ staff with special responsibility 
for purchasing. But we found that, in most cases, the investigators carrying out 
research, who also make the equipment purchases, were not obliged to consult 
them. In some cases they were not even obliged to notify purchasing staff of major 
purchasing activity. Our examination revealed that the universities used the 
expertise of specialist procurement skills in only nine out of the 68 cases we 
studied. In the majority of the cases where they were used, they were asked to 
provide advice on compliance with the European Community procurement 
directives, rather than on how to obtain the best possible price. 


A whole life costing approach to equipment procurement 
is not used 


E2274 Procurement decisions which are made on the basis of the initial cost of an 
item, may not yield best value for money in the long run. The Joint Procurement 
Policy and Strategy Group's guidance on whole life costing, recommends that it 
should be applied, where appropriate, so that the longer-term financial 
implications of various options can be fully recognised. Whole life costing takes 
account of the total cost of an item from determination of need right through to its 
eventual disposal and replacement. Its use will identify the full cost of acquiring 
equipment, its on-going running costs and all other significant costs. It can also 
enable better comparisons to be made between different equipment models and 
assist tender evaluations. It also encourages users to specify from the outset how 
the assets will be used, determine technical specification and establish what 
long-term support and maintenance is likely to be required. We found that none of 
the examples examined used the whole life costing approach either at the grant 
application or procurement stages. This was mainly because this costing 
technique is relatively new in higher education and guidance on its use was not 
issued until October 1998, after we had completed our fieldwork. 


Alternatives to procurement are not always considered 


P25] All possible solutions relating to the perceived need for research equipment 
should be investigated before a decision is made to purchase new equipment. 
Alternatives to procurement include, sharing existing equipment already held by 
research departments, negotiating the loan of suitable equipment from a supplier 
or by purchasing good second-hand or ex-demonstration models. Alternative 
methods of acquiring equipment should also be considered. For high value 
research equipment this can include, for example, leasing agreements, 
outsourcing or joint ventures. We found some examples where ex-demonstration 
models had been obtained (Case 12) and where equipment had been loaned from 
suppliers. However we found no evidence of leasing being actively considered. 
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Illustrative case 12 
Alternative option taken 


Purchase of a Mass Spectrometer (an instrument used for measuring the index of defraction). The 
equipment was essential to the efficient identification of key structures, including the analysis of high 
molecular weight compounds. By purchasing an ex-demonstration model the university managed to 
obtain a discount of around 55 per cent off the full list price. 


Ways of securing improvements 
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PT We conclude that gains can be secured by improving the efficiency of 
equipment procurement. This can be achieved if the universities are encouraged 
to use the basic best practice guidance recently set out in the "Equipment 
Procurement" guidelines for the higher education sector. One way of encouraging 
the adoption of best procurement practice would be for the Council to make 
adherence to the guidance a condition of grant. The Research Councils and the 
Funding Councils should work together to improve procedures in the light of the 
advice on best procurement practice that has been developed by the Government, 
the Funding Councils and the sector representative bodies. Ways should be sought 
to obtain assurance that best procurement practice is being used, and measures 
should be established to monitor progress in equipment procurement 


performance. 
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Part 3: Arrangements for approving and 
reviewing grants 


EX@ = This part of the report shows that the Council's arrangements for obtaining 
value for money for grants have, until now, been focused on reviewing the scientific 
case for equipment made in the grant application. Once the project is completed, 
the Council review the quality, output and cost effectiveness of research and 
reconcile the expenditure against the funds awarded. Relatively little has been 
done to monitor procurement outcomes specifically, such as whether the 
equipment funded was actually purchased, and if so at what price, and whether 
good procurement practices were used. 


E¥1 = This part sets out: 


HM how universities identify equipment needs and sources of research 
funding; 


M@ the arrangements for inviting grant applications; 
HM what the application for grant contains; 


HM how the Council review the applications, award a grant and check 
outcomes; and 


HM the monitoring of the usage of research equipment. 


The universities should identify and justify their research 
equipment needs and the appropriate sources of funds 


EX] The seven universities that we visited did not have a reliable database of all 
the research equipment they possessed and did not have an overall plan setting out 
what their future needs might be. Most university heads of department did, 
however, have a clear idea of the major items of laboratory and research 
equipment that they would purchase if funds were available. 


EZ] It is the role of the Higher Education Funding Council to provide 


universities with the buildings, equipment and other infrastructure to establish 
and maintain "well found" research laboratories. The Research Councils have 
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traditionally funded the additional equipment necessary to carry out particular 
research projects they wished to support. The individuals we visited at the 
universities told us however that, because of pressure on the funds provided by the 
Higher Education Funding Councils, they were looking to Research Council grants 
to provide some of the basic equipment they needed as well as equipment 
dedicated to specific research projects. The universities also told us that there was 
therefore a considerable incentive, moderated by the time and effort involved, to 
submit as many grant applications as possible to the Research Councils to secure 
this funding and to err on the generous side with their equipment estimates. 


The Council invite and review applications for grant 
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EX] Until 1995 the Council invited applications to be submitted by two 
deadlines set each year, but now they can be submitted at any time. 


EX3 In 1997-98 the Council received over 5,000 research grant proposals for 
consideration. Standard grant proposal forms are completed by the principal 
investigators who should provide, among other things, a case for support of the 
research proposed, including justification of the resources required and a 
summary of the proposer's track record. Proposals should itemise all single 
equipment items estimated to cost more than £1,000 and identify separately 
equipment estimated to cost more than £250,000 as large capital items. The latter 
are subject to special payment arrangements, if funded. The Council provide no 
specific guidance as to what the justification of equipment resources should 
include and what supporting information should be provided. 


Exd Our examination showed that the extent and quality of justification 
provided in support of proposals varied. In the best cases, researchers met all the 
criteria for justifying equipment resources. They explained: 


M@ why they needed the type of equipment requested; 
M@ the performance specification they required; and 


M the potential sources of supply. 


They provided estimates from a representative number of suppliers, appraised the 
merits of supplier proposals and justified their preferred choice. In the worst cases 
researchers just indicated what they intended to buy without substantiating their 
choice of supplier or their estimate of cost. 
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EX} As shown in Figure 5 the primary mechanism for assessment of proposals 
is peer review by experts in the proposed field of research. Referees are required to 
comment on the scientific merit of the proposal, whether the resources requested 
have been justified and whether the proposal is likely to offer good value for 
money. The guidance provided to them on justification of resources is however 
limited and, in contrast to the guidance for assessing scientific merit, provides no 
criteria to be applied when making these judgements. 


EX) Our examination showed that the quality of individual peer reviews in 
respect of the justification of resources and expected value for money was patchy 
and that negative comments about equipment prices were rarely substantiated by 
hard evidence. Sometimes, there also appeared to be a contradiction between the 
comments made by reviewers and decisions taken. For example we found eight 
cases where, despite some negative comments, the proposals were fully funded. In 
seven cases we found that grants had been reduced where the referees had not 
made negative comments. It is however the function of the assessment panel to 
weigh up the comments by individual referees and reach conclusions, sometimes 
in the face of conflicting reports, on the merit and cost effectiveness of grant 
proposals. 


ESt) We encountered only one case where deficiencies in justification of 
equipment resources was pursued in a robust way. In that case the grant was 
restricted to 50 per cent pending receipt of information about specification, choice 
of equipment and supplier and pricing. This information was produced within four 
days and the equipment was subsequently fully funded. The Council has recently 
adopted the practice of sending referees' comments anonymously to applicants to 
give them an opportunity to respond to any issues raised. This practice should lead 
to better justification of the research resources sought. 


Equipment typically costs 16 per cent less than the amount 
awarded for its purchase 


ERE® We encountered only two cases where prices were obtained through formal 
competitive tendering at the proposal stage. The universities told us that, in most 
cases, it was not practicable to go to competition or demand best prices at the time 


of their grant application because: 
HM grant approval was not assured at that stage. The actual success rates 


achieved by the universities visited by the National Audit Office were 
generally greater than the average success rates for universities overall 
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and the one in ten chance they believed to be the norm. Although success 
rates vary, the average chance of a proposal being funded by the Council is 
around 38 per cent overall (Figure 11) 


Ane icetorteuerees SFIgue 1p 
rates of universities 
visited The success rates, at the universities visited by the National Audit Office, tended to be better 
than the average for all universities. 
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Source: The Engineering and 

Physical Sciences Research 

Council’s research grant 
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M While the application is being appraised the price might change. 
Applicants told us that they anticipated that it would take at least six 
months between submitting the application and receiving the grant and 
sometimes longer. For the grants we selected, elapsed time ranged from 
two weeks to 62 weeks, with a average of some 24.7 weeks. For all grant 
applications handled by the Council the average elapsed time between 
receipt of grant proposals and notification of outcomes has fallen in each 
of the last four years, from 19 weeks to 16 weeks (Figure 12); 
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Pinesea tite [Figure 12 


The average elapsed time between receipt of grant proposals and notification of outcomes 
has fallen in each of the last four years. 


1994-95 


1995-96 | 18 weeks 


10 15 20 
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M In addition the applicants we interviewed admitted that they sometimes 
inserted as high a figure as possible in the proposal. Many regarded the 
grant proposal as an opportunity to maximise research equipment 
funding. They knew that, ifthe project budget was approved at an inflated 
level, they would be able subsequently to negotiate lower prices and 
obtain more equipment with the money available. Sometimes they also 
used this device to minimise the impact of reductions in equipment 
budgets, which might be imposed during the grant review process. In 
some cases we found that Value Added Tax was included in grant 
applications for items which were subsequently found to be zero rated. 


ERr) For our selection of 68 equipment items, we took the estimated cost in the 
research application, on which the grant was based and paid and compared it with 
the actual price. In one case the item has not yet been purchased. This showed 
that, on average for our selection of cases, equipment cost 16 per cent less than the 
estimate and that in: 


M 46 cases expenditure was less than estimate by on average more than 
26 per cent. For example see cases 13 and 14; 
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M 11 cases costs were higher (in four cases where the variation was highest 


it was because different equipment was purchased to that specified in the 


application); and 


M@ in ten cases the estimate equalled expenditure. 


Illustrative case 13 
Poor assessment of needs and costs 


One research department identified a need 
for seven workstations and six personal 
computers at an estimated cost of £85,000. 
The Council considered that there was only a 
need for three workstations and three 
personal computers and provided £67,200. 


Illustrative case 14 
Poor assessment of needs and costs 


One university estimated the price of a "time of flight’ 
spectrometer (an apparatus used for separating 
isotopes, molecules, and molecular fragments) at 
over £95,000. The final expenditure was £60,000 
for two spectrometers of lower specification than 
originally proposed. The equipment was needed for 


The department reassessed their computing research into biocatalysis, nanostructured 
requirements and decided to go for a materials, electrocatalysis and polymer research. 
personal computer approach. They 

subsequently purchased seven personal 

computers for £15,000. 


ESF] In the two cases in which the initial estimates were based on competitive 
quotations, the estimates proved more accurate. In these cases expenditure was 
within 10 per cent of the estimate, compared with the 26 per cent variance arising 


from non-competitive quotes. 


ER?Y Researchers acknowledged that the accuracy of estimating could be 
improved by either: 


®@ ensuring that initial estimates were based on evidenced prices preferably 
obtained through competition but at least from several potential 


suppliers; and 


M@ post award of grant, requiring them to resubmit prices following active 
negotiations with suppliers or competition. 


ESF] One university purchasing officer suggested that the Council could 
incentivise the process by sharing with the university savings achieved through 
price negotiations. Nearly all the researchers we interviewed accepted that there is 
at present no incentive for them to be more precise in their estimating, other than 
ensuring that the level of the grant proposal is pitched at a level they consider will 
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be accepted by the Council. They also told us that the universities use the excess to 
fund additional equipment for the benefit of the research project being 
undertaken. 


EST The lack of evidence submitted in support of equipment cost estimates 
increases the risk of the grant being awarded for overestimated costs. We consider 
that better evidencing of research equipment costs, where the information can be 
provided with appropriate use of resources, would be one element of providing 
better quality grant applications and would help ensure applicants had a greater 
chance of success. 


A number of equipment funding requests are reduced 
by the Council 


A number of grants are lower than the amount requested. This applies 
particularly to equipment. Thirty two out of our 68 cases had experienced 
reductions in the equipment budgets sought, ranging from four per cent to over 
80 per cent. Such reductions are usually introduced following peer review where 
the equipment budgets are perceived to be too high or where referees consider that 
universities should obtain additional funding from other sources. Occasionally the 
Council may reduce the funds offered because of budgetary constraints. In none of 
those cases did we see evidence that the Council had gone back to the applicant to 
assess whether the additional funds could be found and, if not, what impact this 
would have on the research work. In those cases where the Council reduced the 
funding for a long list of research equipment items, it was left for the researcher to 
decide whether to accept or reject the reduced offer of funding and to prioritise the 
items he wanted to buy with the reduced budget. 


ESf) The Council recognise the problems that reducing grants can introduce and 
are taking steps to improve the situation. In the year ending July 1998 they found 
that grants were reduced by 6 per cent (by value) overall and by 15 per cent (by 
value) for the equipment element of the grant. The Council also told us that they are 
taking steps to encourage higher education institutions to submit fewer, higher 
quality proposals, as one measure to minimise the number of grants they reject or 
reduce. The Council have also introduced a procedure to allow investigators to 
comment on referee's comments in advance of a decision which provides an 
opportunity to counter unjustified recommendations for reductions in equipment 
funding. But we consider that these various steps have not yet fed back 
significantly into the perceptions of grant applicants. 
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The Council monitor and evaluate the results of research activities 
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Est) As part of their monitoring and evaluation of programmes the Council 
evaluate, at the end of the grant period, the outcome of the research project and its 
contribution to the Council's overall programme and mission objectives. They 
place emphasis upon the cost effectiveness of the research and the value for money 
obtained. 


Eft) They grant reasonable freedom and flexibility to researchers to manage 
resources according to the needs of individual projects. Sometimes they require 
them to submit periodic reports, attend evaluation meetings or require their 
programme staff to monitor the research project. In 1997 the Council, in 
collaboration with the other Research Councils, introduced a programme of 
periodic "dip-stick testing" at universities in receipt of research grants mainly to 
assess the administration and accounting procedures applied to research grants. 
This does not currently cover testing of procurement practices and examination of 
individual equipment procurements. 


ERa8 At the end of the grant period the university is required to submit a final 
expenditure statement identifying all expenditure incurred against the project 
compared with the payments made by the Council. The Council reconciles the 
expenditure against the grant funded and makes a final payment of any balance 
due, or recoups any overpayment made. The applicants are also required to 
submit a scientific report setting out the results of the research activity and 
detailing the use of the resources, including transfers between heading and major 
purchases. Both the final expenditure statement and the scientific report must be 
sent to the Council within three months of the end of the grant. The Council applies 
financial penalties for late or non-submission of final expenditure and final 
reports. 


Ef?) The information that is provided by the researchers to the Council on 
equipment expenditure in general and on individual purchases is not compared to 
the equipment originally requested. The Council acknowledge that they need to 
correlate more closely the terms of the award with the expenditure actually 
incurred, the research activity and the outcome. They told us that, with effect from 
April 1999, they signed a Framework Agreement with individual universities, 
which set out the duties and responsibilities of all parties in relation to research 
grants. 
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E®<] The scientific reports produced at the end of the grant funded activity are 
assessed by peer review in much the same way as the original proposal. As part of 
this process referees are required to comment on the use of resources. Most of the 
applicants we interviewed in relation to our selection of grants also act as referees 
for other grants. They told us that they often felt unable to comment on this aspect 
as they do not have all the relevant grant information and financial statements at 
their disposal. 


EZ] Our examination of a selection of grants showed that, in 16 cases 
(24 per cent), the equipment purchased was not what was specified in the 
proposal. We also noted that there was no requirement for the applicant to inform 
the Council if equipment needs changed. Investigators told us that they often 
firmed up on what equipment they wanted and its specification only after their 
grant was approved. This is particularly sensible for the acquisition of computer 
equipment, where the market is very fast moving, but we also saw it in other cases. 
Our examination revealed one instance where equipment was purchased in 
anticipation of the Council's approval (Case 15) which is permitted within the 
Council's grant conditions provided the researcher seeks retrospective approval. 
We also found another instance where both the research and the equipment was 
different to that proposed (Case 16), a situation that was subsequently picked up 
under the Councils final report review procedures. 


Illustrative case 15 Illustrative case 16 

Purchase in anticipation of approval Unauthorised change in research 

The Council reduced the equipment funding In one project, both the equipment and the research 
requested in a rolling grant proposal in was different to that originally specified in the 
respect of the purchase of a £40,000 laser proposal. Computers were purchased for £31,000 
which they felt was not justified. They agreed instead of a stereo camera at £32,000. The Council 
to review their decision at the mid-term were not informed of these changes until the project 
review of the grant two years later. The had been completed. 


university exercised the flexibility permitted 
in their management of the grant resources 
and purchased the equipment from the 
grant. They felt it was essential for their work 
and intend to justify the purchase at the 
review stage. They are confident that 
retrospective approval will be granted but 
acknowledge that, in the unlikely event it is 
not, the university would be obliged to fund 
the purchase from their own resources. 
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Outturn expenditure 
pattern on equipment 
costs for completed 
projects 


Source: National Audit Office 
database of completed 
projects 


E&=a We analysed the 2,498 grants, completed during the two years ending 
31 March.1998, which included funding for equipment. Overall, the final 
expenditure equalled the funding originally approved, although individual 
equipment grant out-turns varied considerably. Over 70 per cent of the grants 
were within plus or minus ten per cent of the approved equipment grant 
(Figure 13). 


Figure 18 


Over 70 per cent of grants were within plus or minus ten per cent of the approved equipment 
budget. 
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Effective monitoring of equipment usage is rarely undertaken 
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E®%4 It is important for equipment to be justified in terms of usage as well as 
technical performance at the application stage. We observed that, until recently, 
the Council had asked universities for estimates of usage to accompany proposals 
for equipment costing over £100,000. But this practice has now ceased. From our 
selection of grants, planned usage ranged from 40 hours a week with the 
equipment in use 40 per cent of the planned time to 150 hours a week with the 
equipment in use all of the time. 
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After the research equipment has been purchased, its effectiveness can 
often best be judged through an evaluation of how much and how well the 
equipment has been used and managed. To monitor such performance it is 
essential that major items of research equipment are recorded and mechanisms 
are in place to record and permit the monitoring of usage. From our selection of 
grants we found that formal recording and monitoring of usage was rarely 
undertaken. For those research equipment items costing over £50,000, only four 
cases (12 per cent) were found where effective monitoring was being undertaken, 
sometimes through detailed recording of real time usage data embedded in the 
equipment itself. In 13 cases reliance was being place on the equipment booking 
sheets without any use being made of the data, and in 20 cases we found that usage 


was not actively considered. 


Es) The Council have not taken steps to highlight the importance of maximising 
the usage of research equipment. We recognise that some research equipment 
purchased for very specific and specialised research may not be fully used even 
though it achieves the objectives of the project. In other circumstances there may 
be limited scope for sharing. But, in the majority of instances scope is likely to exist 
for increasing usage or at least demonstrating actual against planned usage. We 
found that, in the majority of cases, actual usage versus planned is not monitored 
by either the Council or the universities. We also found that monitoring and 
evaluation of completed research projects by the Council has never included 
reviewing the usage of the equipment funded. 
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Appendix 1 


Selection of case studies, audit 
methodology and results 
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Our examination was intended to identify those aspects of the Council and 


university practice relating to the procurement of research equipment which 


represented a risk to value for money. We commissioned independent consultants, 


Société Genéralé de Surveillance, to price check 68 research equipment items. Our 


consultants were asked to see whether the prices paid for the equipment were in 


line with market levels and to identify whether there was scope for purchasing the 


equipment more cheaply. 


This appendix sets out: 


why we focused on the Engineering and Physical Sciences Research 
Council rather than the other five Research Councils; 


why we selected certain universities at which to carry out our 


examination; 


the basis on which we selected grants for case study; 


the basis on which we selected individual items of equipment for price 
checking; 


what we examined; 


what we found; 


what this means for the Council; and 


how this affects the other Research Councils. 
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Why the Engineering and Physical Sciences Research Council? 


Distribution of research 
grants at 31 March 1998 


Source: National Audit Office 
database of live grants 


We reviewed the systems by which all six Councils administer their research 
grants and found that they were very similar. We concluded therefore that an 
in-depth examination at one Council would be more likely to produce convincing 
results than a less detailed examination of all Councils. We focused our 
examination on the Engineering and Physical Sciences Research Council (the 
Council) as they are the largest single provider of grants among the six Research 
Councils. Their grants represented around 41 per cent by value of the total live 
grants for all six Councils and 56 per cent in respect of their funding of equipment 
(Figure 14). At the end of March 1998 the Council had around 4,000 live grants in 
payment valued at about £675 million of which some £100 million was intended 
for the procurement of research equipment. 


Sgt 4 


EPSRC are the largest individual funder of research grants overall and fund more research equipment 
than the rest of the research councils put together. 


Value of grants Number of higher 
Councils Number of Total grants Equipment budget education institutions 
grants receiving grant 
£m % £m % 
EPSRC 4,057 674.4 : {2 
Other five councils 4,813 952.1 58.5 77.9 43.5 105 
Total 8,870 1,626.5 100 WANS 100 


Why were seven universities chosen? 


Universities visited by the 
NAO 


Cambridge Sheffield 
Glasgow Surrey 
Imperial College Warwick 
Liverpool 


We found that a comparatively small number of universities receive over half of the 
grant provided by the Council to universities (Figure 15). So as to be able to 
examine a smaller number of cases in depth, we focused on those universities 
which had received the largest proportion of grants. We examined the 
procurement of research equipment at six from the top fifteen universities and at 
one other from the top twenty (Figure 16). We chose one from outside the top 
fifteen in order to ensure that the same procurement arrangements and results 
applied to both the larger and the smaller universities. To reduce the 
administrative burden on individual universities we avoided those which had been 
visited by the Councils as part of their own programme of grant monitoring or were 


planned for a visit by them. 
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Distribution of EPSRC 
research grants between 
higher education 
institutions 


| Figure 1 


Fifteen universities receive 55 per cent of EPSRC research grants and 57 per cent of the equipment 
budget funded through grants. 


Institutions Research grants Grants with equipment budgets 
Number Value (£m) % Number Value (£m) % 
Major eel BAG , 
Others 108 45.5 1,504 43 
Total 123 674.4 100 2,931 101.6 100 


On what basis were the individual grants chosen for case study? 
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We considered that 8 to 12 case studies per university would allow coverage of all 
the characteristics listed below. Our consultants, Société Genéralé de Surveillance, 
also advised us that this was a reasonable number of items for them to price check 
in the time envisaged. We chose our cases to reflect: 


M = where the major expenditure was being incurred. This led us to focus on 
higher value grants because 10 per cent of higher value grants accounted 
for 60 per cent of total value; 


M the areas of research being supported by the Council and the university 
departments working in these areas; 


M coverage of both mature and more recent grants; 


M@ the whole range of equipment grant values - from less than £50,000 to 
more than £500,000 (Figure 18); 


M the various types of grants covering for example, single grants, rolling 
grants and special equipment initiatives; and 


M@ different types of equipment. 


The number of cases finally selected which enabled us to meet our various criteria 
was 68 (Figure 17). Because the number of criteria we wished to cover was 
extensive we decided against selecting the cases on a random basis. The degree of 
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stratification (making sure enough cases with each characteristic are chosen) 
would have significantly increased the numbers requiring selection and thus 
increased the burden on the universities. 


Selection of research SEGRE 


grants for case study 


examination We focused our selection on those universities receiving most research funding. 


Value (£m) 
Institutions Universities Research grants Grants Equipment budget 
Top fifteen 6 60 
Others 1 8 
Total ZL 68 60.26 14.02 


Distribution of EPSRC Figure 18 


equipment budgets and 


NAO selection We selected mostly from those research grants with the most significant equipment budgets and also 


covered the full range of grant values. 


Value range EPSRC Equipment budgets NAO selection 
(£m) No. (£m) No. 
Less than £50,000 38.86 2,530 0.38 
£50,000 - £100,000 16.10 236 Gee 
£100,000 - £200,000 NZ 99 2.07 
£200,000 - £500,000 122: 45 2S 
Over £500,000 20.33 21 7.10 
Total 101.63 2,931 14.02 


On what basis were individual items of equipment selected for 
price checking? 


For each grant chosen for case study examination, we selected an item of 
equipment purchased from the grant which we considered would be capable of 
independent price checking. This necessitated examining the grant and 
accounting records of the Council and the universities to identify items of 
equipment actually purchased and charged to the grant. 
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What did we examine? 


< 


The principal elements of our methodology for examining the grants were: 


For the grant administration procedures and university 
procurement arrangements: 


review and analysis of the procedures used by the Council for receiving, 
reviewing and awarding grants for research equipment; 


review of their grant monitoring and evaluation of final results; 


comparison of their practices with those of the other Research Councils; 
and 


visits to the institutions to discuss their equipment purchasing and 
accounting procedures in general and in particular their acquisition of 
research equipment with Council funds. 


For the case study selection: 
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examination and analysis of Council documentation relating to the case 
studies selected at seven universities (Figure 17); 


visits to the individual researchers at universities to examine how they 
had justified the resources requested, what they had purchased with the 
grant awarded and how the purchase process had been managed; and 


an independent price check by the Société Genéralé de Surveillance of 
individual items purchased with the above grants to confirm the price 
paid or to identify those cases where scope had existed for savings to be 
achieved. 


What did we find? 


Analysis of equipment 
costs against estimates 
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Overestimating 


We found, in relation to our 68 case studies, that in 46 cases (nearly 70 per cent) the 
equipment funded was actually purchased for less than the estimated cost on 
which the grant for equipment was assessed (Figure 19). Overall the equipment 
was purchased for 16 per cent less than estimated. 


[Figure 19 ua 


Overall the equipment was purchased for 16 per cent less than estimated. 


Equipment expenditure Number % variation 
Less than estimate 46 - 26 
More than estimate 11 + 11 
Equal to estimate 10 0) 
Item not yet purchased 1 0) 
Total 68 -16 


Scope for better procurement 


We provided 68 equipment items, purchased for a total of £8.7million, to our 
consultants for price checking. This proved not possible in 13 of these cases. In six 
of the 13 cases the item was not suitable for price checking because it was part ofa 
larger equipment built specifically for the universities and in seven other cases, our 


consultants encountered a lack of co-operation from suppliers. 


Where a price check was concluded, our consultants confirmed that the items had 
been purchased well in 20 cases but identified scope for savings which had not 
been achieved in 35 cases. Figure 20 shows that scope for savings existed across 
all ranges but that the incidence of savings in relation to the numbers examined 


was generally greater in the higher value ranges. 
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Analysis of results by 
grant values 


Analysis of results by 
catagories of potential 
savings 


Fae 2 


The results of our price check revealed that scope for savings arose across all the value ranges of grants. 


Value range of grant Number Results of price check by consultants 
selected Not Purchased Cases where savings apply Percentage 
possible at keen Price Competition savings 
price check 

Less than £50,000 10 4 2 4 8 6.4 
£50,000 - £100,000 2S 3 10 12 22 Wee: 
£100,000 - £200,000 15 3 6 6 12 5.0 
£200,000 - £500,000 10 1 1 8 6 5.9 
Over £500,000 8 2 1 S 3 7.8 
Total 68 13 20 35 51 6.9! 
Note: 1. These potential savings exclude the estimated savings that could be obtained from 


co-ordinated purchases. 


Figure 21 shows that the highest potential savings, in percentage terms, related to 
the cases where there had been failure to co-ordinate high value purchases. The 
largest number of potential savings arose from failure to pursue competitive 
purchasing and not obtaining available discounts. 


| Figure 21 eee 


This shows an analysis of the results of the National Audit Office examination and their consultants’ 
price check. 


Category of potential savings Number' Scopefor Average%  % for 
savings of price items 
identified (£) paid © examined * 


Price check not feasible (mainly build items) 6 - - = 
Price check not achieved by consultants 7 - = - 
Items purchased at keen price 20 - = 
Competition not sought 48 248,400 5 }6.9 
Prompt payment and other discounts not sought 85) 344,700 6.8 } 
Co-ordination of purchases not considered 2 180,0007 10 2 
Total 773,100 8.9 
Notes: 1. Potential savings for one grant can occur in more than one category. 

7D Relates to two specific items within the NAO selection. 

3; The.total cost of the items examined was £8.7 million. 


The overall results indicate that savings of around nine per cent could have been 
achieved through better procurement. 
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What does it mean for the Council? 


We discussed our emerging findings with a Focus Group who acknowledged that 
our findings were in line with their own experience. The members were: 


Name Title/Organisation 

Peter Maxwell Director of Finance and Administration. } Engineering and Physical 
David Harman Programme Services } Sciences Research Council 
Peter Smith Programme Manager Materials } 

Allan Robertson Procurement Adviser to all the Research Councils 

Steve Butcher Principal Auditor, Higher Education Funding Council for England 

Keith Root Director of Finance, Office of Science and Technology 

Kerry Tregonning Procurement co-ordination, Office of Science and Technology 

Professor Roger Cowley = Clarendon Laboratory, University of Oxford 

Stephen South Director of Procurement, University of Sheffield 


As our case studies were not selected on arandom basis we cannot place statistical 
confidence intervals around our results. But, taken together, the results indicate 
potential savings could be made by the Council and the institutions. We consider 
the findings from our selection indicate savings of a similar size could be made 
across the grants awarded by the Council because: 


a) our selection covered a sizeable proportion of the Council’s equipment grant 
expenditure (£14 million out of £102 million); 


b) our selection covered all types, sizes and characteristics of grant and a range of 
equipment. Savings were found across the whole range of procurement 
situations not just, for example, on one type of equipment at one university; and 


c) university systems and approaches to procurement are broadly similar. 


A saving of 9 per cent through better procurement would represent around 
£3.5 million. Improved estimating, based on the 16 per cent average 
overestimating we identified, would represent some £6 million. 


What does it mean for the other Research Councils? 


The six Councils’ grant systems are very similar and the same institutions are 
being funded. We consider that it is very likely that similar opportunities for 
savings exist in their grant arrangements. The savings which we believe the 
Council could make may also be possible in the other Research Councils, although 
because of difference in the operation of their grants, it is not possible to provide an 
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accurate estimate of the potential savings. If our results apply to the other five 
councils, potential savings of some £3million could have been achieved from 
improved procurement. This figure is based on using the same savings level of 
9 per cent, and applying it to the other research councils’ expenditure on funding 
research equipment (some 44 per cent of the total Research Council expenditure). 
We note that the Research Councils and the Office of Science and Technology have 
formed a working group in conjunction with representatives of the higher 
education sector to explore how best to carry forward the recommendations of this 
report. 
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Reports by the Comptroller and 
Auditor General, Session 1998-99 


The Comptroller and Auditor General has to date, in Session 1998-99, 
presented to the House of Commons the following reports under Section 9 
of the National Audit Act, 1983: 


Ministry of Defence: Modifying Defence Equipment......................0066 HC 24 
Overseas Operations, Governance and Management at 

POUC aN LOT EG EL LULe Meera fake ee Se, Scan Svanadnencccedevdcesntods HC 23 
MUO PIOCAL OM OF Pratl ir dUneeee.ctece creer tte ct ne eos Sasa a waa suai oooh veanncendonaes HC 25 
BBC World Service: New Financial Arrangements....................000ccceeeee HC 26 
Ministry of Defence: Management of Office Space ...................000cec00e HG 105 


The Final Accounts of Chessington Computer Centre, 
Recruitment and Assessment Services and the 


Occupational Health and Satety AGenGy cc. 4 o.cecc sneeee-dacsee ess cenccessess HC 154 
Sales of Residual Loans to Housing Associations..................00ccceeeeeees HGo2 
Public Trust Office: Protecting the Financial Welfare of People 

NRE LONI ea MIVA ACE Vue mut narod meseneaienig Atenas coe Neder «da cobaauyanis daceoee HC 206 
How the Utility Regulators are addressing the Year 2000 Problem 

FEIN CHAN TEES oe le 9. eae Ome ene oes are ee HGrzZ2 
UX Selo ee AT a pick VINTON CLLCTIN Ce ce ere waite ay, son seeds snes sedsaaeelSceseere2end HOZ223 
The NHS in Scotland: Making the Most of the Estate......................... HC 224 
Pang ULrender andl GOMmpelSallOnca-sce.-<.45laccekdecenasencesoscanset HG 220 
Pe riancenl ake NEL COU Le so ODOR aera trnjicayisctcceewennnieenacndersecnceees tGe2o | 
Audit of the Unemployment Assumption for the 

NarGiiylo 9 UO lOCel Er OleCh ONG scese ss agincte-: devoissese dc asethe Sesesvedetins HC 294 
Royal Brompton Hospital — Sale of North Block Site ............. HOZ52 


Financial Management and Governance at Gwent Tertiary College...HC 253 
Audit of the General Budget of the European Union 


TOmstOO and Nelaredr Level DCMCNtS .tetied: ores catc-ccacicrseeos-aenanmaies Her279 
The Immigration and Nationality Directorate’s 

Pee VOL Mme eG PALI ene th et SRC ts ete nid Sacoeoseacn Gunns eS vsnweos HGrz7 7 
ANIC SHEL OOLCT NE) Whey Peat] Big aia Yen s CO) aR, | eee et see ee eee HC 280 
Progress on measures to combat Housing Benefit fraud.................... HG319 
The Private Finance Initiative: The Contract to Complete and 

Operate the A74(M)/ M74 Motorway in Scotland ...................::006 HC 356 
Investigation of alleged irregularities at Halton College... HG 30% 
emt ciel a 10 [16 | ee reemnnee saree eta eet ee ene Mire at Lezwnssntescwenslss anseaue’ HC 358 
DA LOLOL IN a Me OVeD TAS GO A ao teecas see RRet ot agceveaseaeesomestseug oe nnatemene= HG-359 
The PRIME project: The transfer of the Department of Social Security 

ee Ore OMA LSC GLO Wc erecta a cave aaienss uma raeanssvasse se ncacoueseaeys Be s70 
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The 1992 and 1998 Information Management & Technology 


strategies oMthe NHS ExeCutivew os,.:eer mae ..siee tee eae nee piCayale 
Managing Sickness Absence in the Prison Service «..::.........csescse00-e-+- HG 2 
Vehicle Emissionsil esting 22 peer cacn. coe eet aeerer eee eee eee HC 402 
Giving Customers a Choice - The Introduction of Competition 

into,the Domestic Gas .Manket tenance ee ae HC 403 
Arts Council of England: Monitoring major capital projects 

funded by thetNalona LLG Oy. c ee ee ee ccc cree eee HC 404 


The PFI Contract for the new Dartford and Gravesham Hospital....... HC 423 
The Insolvency Service Executive Agency: 


Company Director Disqualification - A Follow Up Repott.............. HC 424 
The Millennium Threats22i,Dayecamd.( Gumi jee ee HC 436 
Procurement in the English Higher Education Sector ................-2...4. HC 437 
The Management of Medical Equipment in NHS Acute Trusts 

TWAT) AROUSAL: 00 RenROR ier MEaDnee Serene CMR Mote t ie pier) kV eASME aedhah AB AER le HC 475 
Scottish Further Education Colleges: Managing Costs.................eeee HC 493 
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